Hilbert Projection Theorem
Let H be a Hilbert space ( inner product space that is complete with respect to the norm induced by the inner product) and M be a finite dimensioal subspace of H. Then for any x∈H, there exists a unique y∈M such that
m∈Mmin∥x−m∥
has a unique solution y. In other words "we can find a unique point in M that is closest to x". If m∗ is the closest point to x in M, then x−m∗⊥M.
Proof: See lecture notes.
Remark: The proof stated that m∗=x1 is the closest point to M. It can also be interpreted as the best approximation of x choosen from the vectors in M. The x2 term is the error in the approximation.
Example: Let V=R2 and M=span{[1,1]T}. Find the best approximation of x=[4,7]T in M.
Solution: We need to find m∗ such that m∗=α[11] and ∥x−m∗∥ is minimum.
(x−m∗)⊥M⟹<x−m∗,m>=0∀m∈M
<x−m∗,[11]>=0
Replace x and m∗ with their values.<[4−α7−α],[11]>=0
Recall that <x,y>=xTy.4−α+7−α=0
α=211
m∗=211[11]
Example: Let x∈V and M=span{v1,v2}. Find the best approximation of x in M.
Solution: We need to find m∗ such that m∗=α1v1+α2v2 that is in the span of M and ∥x−m∗∥ is minimum.
(x−m∗)⊥M⟹(x−m∗)⊥both v1 and v2
<x−α1v1−α2v2,v1>=<x,v1>−α1<v1,v1>−α2<v2,v1>=0
<x−α1v1−α2v2,v2>=<x,v2>−α1<v1,v2>−α2<v2,v2>=0
[<v1,v1><v1,v2><v2,v1><v2,v2>][α1α2]=[<x,v1><x,v2>]
[α1α2]=[<v1,v1><v1,v2><v2,v1><v2,v2>]−1[<x,v1><x,v2>]
m∗=α1v1+α2v2
Example: Let V=R3 and M=span{[1,1,1]T,[1,0,1]T}. Find the best approximation of x=[4,7,2]T in M.
Solution of Linear Equations
Consider the linear equation as,
where A∈Cm×n, B∈Cm×1 and x∈Cn×1 is unknown.a) solution exists: when b∈R(A)
b) solution is unique: when N(A)={0}
#EE501 - Linear Systems Theory at METU